Check out the local museum with displays of Kazakh culture.
Green Garden is a nice place to hang out in the evenings; has cheap beer and soft drinks, and is one of the greener spots in town. Most tourists come in the summer when it is quite brown and dusty in the winter it's snowy!, so green is a rare thing.
It's worth going on hikes up into the nearby mountains; find someone familiar who can guide you there.
It's also worth experiencing Kazakh hospitality in a ger, or white felt round-house. Many people set up gers in their yards in the summer, but you can also go outside of town to see gers in their "natural setting."
